ACEEE Int. J. on Network Security, Vol. 02, No. 01, Jan 201 1 



Wireless Vision based Real time Object Tracking 
System Using Template Matching 

I Manju Jackin 1 , M Manigandan 2 
1 Assistant Professor, Velammal Engineering College, Department of ECE, Chennai, INDIA 

Email: rebejack@gmail.com 
2 Senior Lecturer, Velammal Engineering College, Department of ECE, Chennai, INDIA 
Email: mani.mit2005@gmail.com 



Abstract — In the present work the concepts of template 
matching through Normalized Cross Correlation (NCC) has 
been used to implement a robust real time object tracking 
system. In this implementation a wireless surveillance pinhole 
camera has been used to grab the video frames from the non 
ideal environment. Once the object has been detected it is 
tracked by employing an efficient Template Matching 
algorithm. The templates used for the matching purposes are 
generated dynamically. This ensures that any change in the 
pose of the object does not hinder the tracking procedure. To 
automate the tracking process the camera is mounted on a 
disc coupled with stepper motor, which is synchronized with a 
tracking algorithm. As and when the object being tracked 
moves out of the viewing range of the camera, the setup is 
automatically adjusted to move the camera so as to keep the 
object of about 360 degree field of view. The system is capable 
of handling entry and exit of an object. The performance of 
the proposed Object tracking system has been demonstrated 
in real time environment both for indoor and outdoor by 
including a variety of disturbances and a means to detect a 
loss of track. 



Index Terms — Normalized Cross 
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I. Introduction 

Object tracking is one of the most important subjects in 
video processing. It has wide range of applications 
including traffic surveillance, vehicle navigation, gesture 
understanding, security camera systems and so forth. 
Tracking objects can be complex [1] due to loss of 
information caused by projection of the 3D world on a 2D 
image, noise in images, complex object motion, non rigid 
or articulated nature of objects [2], partial and full object 
occlusions, complex object shapes, scene illumination 
changes, and real-time processing requirements. 

Tracking is made simple by imposing constraints on the 
motion and/or appearance of objects. In our application we 
have tried to minimize the number of constraints on the 
motion and appearance of the object. The only constraint 
on the motion of the object is that it should not make 
sudden change in the direction of motion while moving out 
of the viewing range of the camera. Unlike other 
algorithms [3] the present algorithm is capable of handling 
the entry and exit of an object. Also, unlike [3], [4] , no 
color information is required for tracking an object. There 
is no major constraint on the appearance of the object 
though an object which is a little brighter than the 
background gives better tracking results. 
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There are three key steps in implementation of our object 
tracking system: 

• Detection of interesting moving objects, 

• Tracking of such objects from frame to frame, 

• Analysis of object tracks to automate the disc 
mechanism 

A basic problem that often occurs in image processing 
is the determination of the position of a given pattern in an 
image or part of an image, the so-called region of interest. 
The two basic cases are 

• The position of the pattern in unknown 

• An estimate for the position of the pattern is not 
available 

Usually, both cases have to be treated to solve the 
problem of determining the position of a given pattern in an 
image. In the latter case the information about the position 
of the pattern can be used to reduce the computational 
effort significantly. It is also known as feature tracking in a 
sequence of images [5], [6]. 

For both feature tracking and to know the initial 
estimations of the position of the given pattern, a lot of 
different well known algorithms have been developed [7], 
[8] .One basic approach that can be used in both cases 
mentioned above, is template matching. This means that 
the position of the given pattern is determined by a pixel- 
wise comparison of the image with a given template that 
contains the desired pattern. For this, the template is shifted 
' u' discrete steps in the x direction and v steps in the y 
direction of the image and then the comparison is 
calculated over the template area for each position (u,v).To 
calculate this formalized cross correlation is a choice in 
many cases [5]. Nevertheless it is computationally 
expensive and therefore a fast correlation algorithm that 
requires fewer calculations than the basic version is of 
interest. In this paper we propose a complete real time 
object tracking system which combines template matching 
with normalized cross correlation. 

Related works are discussed in Section II and the details 
about template matching are discussed in Section III. In 
Section IV we discuss about the proposed system. The 
system model implementation is discussed in Section V 
and the results are discussed in section VI. 

II. Related works 

Tracking lays the foundation for many application areas 
including augmented reality, visual servoing and vision 
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based industrial applications. Consequently, there is a huge 
amount of related publications. The methods used for real 
time 3D tracking can be divided into four categories: Line 
based tracking, template based tracking, feature based 
tracking and hybrid approaches. 

Line based tracking requires a line model of the tracked 
object. The pose is determined by matching a projection of 
the line model to the lines extracted in the image. One of 
the first publications in this field was [9] .Recently a real 
time line tracking system which uses multiple-hypothesis 
line tracking was proposed in [10] .The main disadvantage 
of line tracking is that it has severe problems with 
background clutter and image blurring so that in practice it 
cannot be applied to applications we are targeting. 

Template-based tracking fits better into our scenarios. It 
uses a reference template of the object and tracks it using 
image differences. These works nicely for well textured 
objects and small inter frame displacements. One of the 
first publications on template-based tracking [11] was 
using the optical flow method. In order to improve the 
efficiency of the tracking and to deal with more complex 
objects and/or camera motions, other approaches were 
proposed [12], [13]. In [14] the authors compare these 
approaches and show that they all have an equivalent 
convergence rate and frequency up to a first order 
approximation with some being more efficient than others. 
A more recently suggested approach is the Efficient Second 
Order Minimization (ESM) algorithm [15], whose main 
contribution consists in finding a parameterizations and an 
algorithm, which allow achieving second-order 
convergence at the computational cost and consequently 
the speed of first-order methods. 

Similar to template-based tracking, feature-based also 
requires a well-textured object. They work by extracting 
salient image regions from a reference image and matching 
them to another image. Each single point in the reference 
image is compared with other points belonging in a search 
region in the other image. The one that gives the best 
similarity measure score is considered as the corresponding 
one a common choice for feature extraction is the Harris 
corner detector [16]. Features can then be matched using 
normalized cross correlation (NCC) or some other 
similarity measure [17]. Two recent feature-matching 
approaches are SIFT [18] and Randomized Trees [19]. 
Both perform equally well in terms of accuracy. However, 
despite a recently proposed optimization of SIFT called 
SURF [20], SIFT has a lower runtime performance than the 
Randomized Trees, which exhibit a fast feature matching 
thanks to an offline learning step. In comparison to 
template-based methods, feature-based approaches can deal 
with bigger inter frame displacements and can even be used 
for wide-baseline matching if we consider the whole image 
as the search region. However, wide-baseline approaches 
are in general too slow for real-time applications. Therefore 
they are mostly used for initialization rather than tracking. 
A full tracking system using only features was proposed in 
[21]. They rely on registered reference images of the object 
and perform feature matching between reference image and 
current image as well as between previous image and 
current image to estimate the pose of the object. However, 



the frame rate is not very high because of their complex 
cost function. Moreover image blurring poses a problem 
for feature extraction. 

Hybrid tracking approaches combine two or more of the 
aforementioned approaches. Some recent related 
publications include [22], which combines template-based 
tracking and line-based tracking. In [21] the authors 
combine line-based tracking and feature-based tracking. 
Even though these algorithms perform well, the line-based 
tracking only improves the results for a few cases and 
might corrupt the result in the case of background clutter. 
In [23] the authors use a template-based method for 
tracking small patches on the object, which are then used 
for point based pose estimation. Since this approach uses a 
template-based method for tracking it cannot deal with fast 
object motion. 

III. TEMPLATE MATCHING 

Template matching is an important research area with 
strong roots in detection theory [24], [25]. It has obvious 
applications in computer and robot vision, stereography, 
image analysis and motion estimation [24]. Template 
matching in the context of an image processing is a process 
of locating the position of a sub image within an image of 
the same, or more typically, a larger size [26], [28], [27]. 
Template matching can also be described as a process to 
determine the similarity between two images. The sub 
image is referred to as the template image and the larger 
image is referred to as the search area (main image) [25], 
[26], [28], [27]. The template matching process involves 
shifting the template over the search area and computing a 
similarity between the template image and the window of 
the search area over which the template [25], [26], [29], 
[27]. These shifting and computing processes operate 
simultaneously and do the repetition until the template 
image lies on the edge of the search area. For the image 
processing purpose, the most common technique for 
measuring the similarity between two-dimensional images 
is by using cross correlation method. A correlation measure 
is determined between the template and respective 
windows of the search area to find the template position 
which has maximum correlation [25], [30]. Generally, for 
two-dimensional image, the correlation coefficient is 
computed for the entire area on the template image shifted 
on the main image. 

The correlation between two signals (cross correlation) 
is a standard approach to feature detection as well as a 
component of more sophisticated techniques. 

A. Template Matching by Cross Correlation. 

Correlation based matching is a simple, yet very popular 
technique in pattern recognition with images. The basic 
method consists of forming a correlation measure between 

a template T and image I (i®t) and determining the 

location of a correspondence by finding the location of 
maximum correlation. Usually I is much larger than the 
template. 

A common measure of dissimilarity is 



©2011 ACEEE 

DOI: 01.IJNS.02.01.105 



12 



ACE EE 



ACEEE Int. J. on Network Security, Vol. 02, No. 01, Jan 201 1 



^Z(/-r) 2 -Z/ 2 -2E^ + X r2 



(1) 



Where the sum is over pixels in the template T and the 

image I . The value of d' will be small when I and T are 
almost identical and large when they differ significantly. 

J (it) 

Since the term " will have to be large whenever, 'd' 

Y(it) 

is small, large value of " v 7 similarly indicates the 
location of a good match of the template. Notice that 

Y(it) 

is the cross correlation of the two signals I 

and?" . 

The simple cross-correlation is called unnormalized and 
it has several major shortcomings. If a certain pixel in the 
image has an abnormally large value, the 

^ associated with this location is likely to have a 
large value even though no good match exists. And since 
zeros do not contribute to the correlation value, a good 
match containing zeros will possible have a small value. To 
overcome these problems, normalized cross-correlation 
(NCC) is often used instead, in which the measure is 
formulated as 



EE 

x y 


T{x, y)l(x ± u, y ± v) 






XX/ 2 (*±«,y±v) 

x y 



(2) 

In this equation (2), I is an image of image size n x n 
with a search window of a larger image and ^ is a 

template of image size n x n; T ~ * Mt ? anc i Mr [ s the 

average intensity of T .This formula keeps P in the range - 
1 to l,with a value close to 1 indicating a better matching. 
For equation 2 to be effective, a prior accurate 

knowledge of the pattern T [ s required .For the detection 
of detonators, we assume the shape of a detonator is fixed 
and known. However, the orientation of the detonator in 
images can be changing and multiple templates with 
varying orientations have to be used. 
Now let as assume that 

T(x,y) = M(i,j) 
l(x + u 9 y + v)=N(ij) (3) 

With image size as mentioned above for both I and T . 

For a sample image (contains left and right breast 
images), the position, orientation and size need to be 
observed. These criteria are important in obtaining the best 
output for the matching process. Unfortunately, most 
images do not satisfy these criteria. Therefore, some 
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techniques are included for preprocessing purpose to get 
the proper images such as flip image, region of interest 
(ROI) and image binarization. These techniques are not the 
major work for this project. Therefore, these techniques are 
not discussed in this paper. The correlation method is 
implemented using the Fast Fourier transform and modified 
into discrete correlation expression for computer usage. 
Discrete Correlation is a mathematical process of 
comparing one image with another image discretely. The 
resulting image is a two- dimensional expression of 
equivalence. 

Supposedly, the relation between template image and 
search area (main image) is defined as; 



CC 



i=l 7=1 



XZm 2 (u)ZS^(u) 



=1 j=l 
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Let CC=1, then for some constant c, the correlation 
coefficient will result as a value 1, proving the 100% 
similarity between both images. This theory is applied 
together with the discrete correlation expression in this 
system. The equation for the two-dimensional discrete 
correlation is given by [5]; 



M-lN-l 



Out(kj)= ^ ^ Inljm, n)lnl{k + mj + n) 



(7) 



Where Inl and In 2 are the input images and Out is 
the output image. MxN is the dimension of both images 
and possibly they have different dimensions and sizes, 
however, it will filled in with certain pixel values to allow 
the matching process. The Inl and In 2 are called the 
search area and the template image respectively. The 
output will reveal the resultant image and its position of the 



13 



ACE EE 



ACEEE Int. J. on Network Security, Vol. 02, No. 01, Jan 201 1 



greatest match between the search area (main image) and 
the template. 

IV. PROPOSED SYSTEM 

An overview of the proposed system as a finite state 
machine (FSM) is given in Figure 1 .The system starts with 
an initialization phase, and then uses the template-based 
tracking algorithm to track the object. If the template-based 
tracker is unable to recover the pose within a certain 
number of attempts the initialization is invoked again. 

Template-based Tracking Algorithm 

We use the NCC algorithm for template-based tracking. 
The object is tracked using this method until a loss of track 
is detected, 

Given: Input image / ; orientation of the object initialize an 
output array with the same size as the input image 

[1] Set an initial threshold value for thresholding the 

correlation result. Normally, it is set at 0.7-0.8 
[2] Repeat the following steps until the result is 
satisfactory. 

> Construct the correlation filter template T 
using an empirical model method. 

> Perform normalized cross-correlation oil 
with T , and store the result in the output array. 

> Threshold these correlation values. If any 
value is larger than the threshold, set the 
corresponding point in the output array to 1, 
otherwise to 

> If the result is not satisfactory, adjust the 
threshold or the correlation filter template and 
continue 

[3] End repeat [1] & [2] 



OK 



□ 
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Templiil^bnKed Tip eking 
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Figure 1. Overview of the proposed tracking system 

A. Required Information 

In our system we use a textured 3D model of the object. 
This model can either be created manually or semi- 
automatically with commercially available products. One 
point to note is that it is advisable to use the same camera 
for texturing the model and for tracking, because this 



minimizes difficulties due to different image quality and 
image formation conditions. 

For the initialization registered images of the object 
called key frames, are required. They can be created 
directly from the textured model by rendering it from 
different views. If the real- world metric pose is required, 
the correct intrinsic camera parameters have to be 
provided. 

B. Initialization 



When initializing, features are extracted from the current 
image and matched to the features extracted in the key 
frame. The pose can then be estimated from the 3D object 
points and corresponding 2D feature points in the current 
image. Since the tracker is using a textured model of the 
object the accuracy of the initial pose estimation is not very 
critical. If on the other hand the reference templates used 
for tracking were extracted from the current image, the 
precision of the initialization procedure would be a major 
issue, because the quality of the tracking result depends 
directly on the quality of the templates used for tracking. 
Hence we decided to directly use the templates taken from 
the textured model in our system. 

C. Reference Patch Extraction 

The textures of the reference patches, which are required 
for tracking, are taken from the textured model or each 
patch the object is rendered so that the patch is oriented 
parallel to the image plane. It is also important to ensure 
that the relative sizes of the object patches are reflected in 
the size of the rendered patches, since the number of pixels 
in a patch is directly proportional to its importance during 
tracking. Since the pose parameters used to render the 
patches are known, the reference patches can be directly 
extracted from the rendered image. These reference patches 
are then reduced a few times in size by a factor of two to 
create a stack of reference patches at different scales, which 
are used to speed up the tracking in a coarse-to-fine 
approach. 

D. Visibility Test 

Attempting to track patches which are not visible will 
lead to erroneous results. Hence it is necessary to ascertain 
the visibility of every patch. This test is performed by 
rendering the model with OpenCV and using the occlusion 
query extension to test which patches are visible and which 
are occluded. The visibility test is performed for each 
frame using the pose estimated in the previous frame. 
Thanks to the occlusion query extension the visibility test 
can be performed very fast, so that it does not interfere with 
the tracking performance 

E. Loss Of Track 

Determining when the tracker lost the object is important 
in order to switch to the proposed algorithm. In our system 
this is accomplished by computing the normalized cross 

T * 

correlation (NCC) between the reference patch I k and 
the current patch I k after the end of the optimization for 
all visible patches. 
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Let / be the reference image and / the current image. 
Further ' p ' be the pixel coordinates of the pixels in the 
reference image The NCC between two patches is defined 
as: 



Ncc(r k ,i k )= 



(8) 



Where N k is the number of pixels of each patch, ju k and 

* 

jU k are the mean pixels intensities and <J k and <J k their 

standard deviations. If the NCC of a patch falls below a 
certain threshold, it is excluded from the tracking. The loss 
of tracking an object occurs mainly due to the presence of 
either due to fast movement of the object or else due to 
tracking object is out of focus. To avoid the loss of tracking 
we propose a tracking system which could be able of 
tracking the object in 360degree rotation. Basically in this 
paper the tracking of the object is considered only in the 
'x' -direction .In this paper we are not going to deal with the 
mechanical design of the rotational disc mechanism where 
as we would be dealing with tracking the object 
continuously without any loss of the object which is under 
test. Complete control of the tracking system is done 
through wireless interface using RF 433 MHz RF module 
from the remote place which would be discussed in the 
following section in detail. 

V. TEMPLATE MATCHING SYSTEM MODEL 
IMPLEMENTATION 

This proposed system model consists of three major 
subdivisions 

A. Hardware setup 

B. Wireless camera Interface with OpenCV 

C. Decision Making rule for continuous tracking of 
object based on ROI 

A. Hardware setup 

This section deals with the rotational disc arrangement 
of the tracking system. The system utilized with single 
stepper motor to account for the tracking of the object in x- 
direction movement of the object. As presented in Fig. 6 
motor is operated using an electronic stepper motor driver 
circuit. We have used Wireless Transmitter and Receiver of 
433MHZ for transmitting and receiving the data from the 
remote computer and the rotational disc arrangement on 
which the wireless Camera is being mounted. At the 
receiving section the wireless camera would be keep on 
tracking the object about 360 degree rotational controls is 
done by means of stepper motor interface along with the 
microcontroller ATMEL89C52. The received data are 
processed based on the decision making rule which is being 
imposed in the microcontroller. The motor driver circuits 
are in turn controlled by the tracking program through the 
data received from the remote PC based on the information 
of the tracking object is under test. Rotational disc 



mechanism implies the motion of the camera is governed 
by the motor installed horizontally. 

B. Wireless Camera Interfacing 

In order to track the object using Template Matching 
through NCC based approach we have implemented an 
Wireless vision interface model for acquiring the real time 
images from remote place through JK wireless Surveillance 
camera. For tracking the objects based on user defined 
template OpenCV is used. To grab the image we have used 
Zebronics image grabber TV tuner for acquiring the image 
from wireless camera through 2.5GHz video receiver 
module interfaced with PC. To interface the Image grabber 
and OpenCV we have used a Dynamic Link Library files 
vcapg2.dll. To track the object completely an efficient 
decision making rule is being implemented by software 
approach i.e., if-else and then conditions based on the 
movement of the object on 'x' -directions. 

C. Decision Making Rule for Object Tracking 

Our object tracker module generates the trajectory of an 
object over time by locating its position in every frame of 
the video. In our implementation the tasks of detecting the 
object and establishing correspondence between the object 
instances across frames are performed simultaneously (i.e 
multithreaded). The possible object region in every frame 
is obtained by means of our object detection module, and 
then our tracker module tracks the object across frames. In 
our tracking approach the object is represented using the 
Primitive Geometric Shape Appearance Model i.e., the 
object is represented as a rectangle. 

The object tracking module takes the positional 
information of x & y co-ordinates of the system of the 
moving object as an input from the object template module 
based on the ROI. This information is then used to extract a 
square image template (whenever required) from the last 
acquired frame. The module keeps on searching it in the 
frames captured from that point. Whenever found it 
displays a red overlaid rectangle over the detected object. If 
the template matching doesn't yield any result (signifying 
that the object has changed its appearance) a new template 
is can be selected based on the user interest on the real time 
video sequence. 

Further on the (x, y) co-ordinates of the tracked object 
are calculated and passed on to the rotational disc module 
from the PC through wireless interface for analysis and 
automation. 

D. Algorithm for the Tracking Module based on Decision 
making rule: 

i. Get the positional information image I(x ,y) frame 
size of the object it is generally considered with 
320x240 

ii. Generate a image template T with size as mentioned 
in Section III A by extracting a square image from 
the last frame grabbed by the camera. The template 
is extracted in the form of a square whose image 
coordinates are given by 
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-Top, Left corner: (x - u, y - v) 
-Top, Right corner: (x + u, y - v) 
-Bottom, Left corner: (x - u, y + v) 
-Bottom, Right corner :(x+u, y+v) 

iii. Search the generated template T in the last frame 
grabbed by the camera. This is done by using an 
efficient template matching algorithm based on 
NCC. 

iv. IF the template matching is successful transmit the 
data through RF Transmitter 433MHz through 
serial Port 

ELSE IF the tracker has NOT detected motion of 

the object then stop automation of the wireless 

camera module setup. 

AND the detector has 

THEN goto STEP 1 (get a new template) 

ELSE goto STEP 5 (get the x, y position) 

ELSE goto STEP 1 (get a new template) 

v. Obtain the position T(x, y) of the match and pass 
it on to the rotational disc x-direction automation 
module for analysis. 

vi. Goto Step iii. 

The below Table I shows the complete details how the 
object would be tracked based on the information of the co- 
ordinates and exactly serial data for controlling the 
Rotational Disc Camera Mounting setup through Wireless 
RF 433MHz. 



TABLE I. 



Automation Of Tracking Module And Its Control Word 



Object 
Tracking 

module 
Direction 


Automation of the Tracking 
system Based on the Co- 
ordinates of the Template 
Matching in the Frame 


Wireless Serial Data 
Transfer Based on 
Coordinates of the 

Object Present to the 
Receiver Section 


LEFT 


Move the camera towards 
direction of the object by 
rotating the stepper motor in 
Anti Clockwise Direction 


'R' 


RIGHT 


Move the camera towards 
direction of the object by 
rotating the stepper motor in 
Clockwise Direction 


'L' 


IDEAL 


Stop the movement of the 
actuators 


'S' 



VI. RESULTS 

Results are presented of the experiment performed using 
the tracking Finite State Machine as shown in the Figure 
l.The experiment was conducted for various images 
acquired from Wireless pinhole camera interfaces with 
Rotational Disc Mechanism for through low cost frame 
grabber Zebronic TV tuner card programmed using 
OpenCV. 

The template matching program written in VC++ 
environment along with some predefined function available 
in OpenCV environment. To make the system 
more versatile for various environmental conditions we 
tried to implement a concept of software approach based 
threshold adjustments which would be helpful for better 
template matching based tracking. 




Template Image- Vehicle 




Figure 2. Template Matching based Vehicle Tracking Prototype with 
real time images of frame size 320x240 



Template Image-Face 




Figure 3. Template Matching based Face Tracking 
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Template Matching-Survelliance Camera 




Figure 4. Template Matching based Surveillance Camera Tracking 
with real time images of frame size 320x240 to move the stepper motor 
actuator towards Left side- Clockwise Direction 




Figure 5. Template Matching based Surveillance Camera Tracking with 
real time images of frame size 320x240 to move the stepper motor 
actuator towards Right side -Anticlockwise Direction 



When the sequences of images is within the region then the 
tracking is smoother where else when the object being 
tracked is out of focus then the camera should be atomized 
through stepper motor so that the object being tracked 
would not be missed out which is being discussed in the 
above section TABLE I. In the result we have consider 
three objects Vehicle prototype, Human Face, Surveillance 
camera movement with reference template match compared 
with actual images tracked output frames are shown in the 
Figure 2,3&4,5 respectively 

Conclusions 

In this paper we have implemented the template 
matching based on NCC algorithm with real time tracking 
under non-ideal environment irrespective of the lighting 
conditions. The system has been automated using rotation 
disc camera mount mechanism setup which is synchronized 
with the algorithm. Such an automated object tracking 
system can be used in applications Defense application 
such as Automatic target tracking, Autonomous target 
hitting through turret by classifying the templates whether 
it is enemy objects or not. Future focuses on tracking 
multiple objects at the same time as well as on improving 
tracker accuracy during camera motion. Even stereo vision 
based tracking system can be carried out for maximum 
coverage of the objects and also to track multiple objects 
too. 
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